Goto

Collaborating Authors

 explanation strategy


From latent factors to language: a user study on LLM-generated explanations for an inherently interpretable matrix-based recommender system

Manderlier, Maxime, Lecron, Fabian, Thanh, Olivier Vu, Gillis, Nicolas

arXiv.org Artificial Intelligence

We investigate whether large language models (LLMs) can generate effective, user-facing explanations from a mathematically interpretable recommendation model. The model is based on constrained matrix factorization, where user types are explicitly represented and predicted item scores share the same scale as observed ratings, making the model's internal representations and predicted scores directly interpretable. This structure is translated into natural language explanations using carefully designed LLM prompts. Many works in explainable AI rely on automatic evaluation metrics, which often fail to capture users' actual needs and perceptions. In contrast, we adopt a user-centered approach: we conduct a study with 326 participants who assessed the quality of the explanations across five key dimensions-transparency, effectiveness, persuasion, trust, and satisfaction-as well as the recommendations themselves. To evaluate how different explanation strategies are perceived, we generate multiple explanation types from the same underlying model, varying the input information provided to the LLM. Our analysis reveals that all explanation types are generally well received, with moderate statistical differences between strategies. User comments further underscore how participants react to each type of explanation, offering complementary insights beyond the quantitative results.


Triadic Fusion of Cognitive, Functional, and Causal Dimensions for Explainable LLMs: The TAXAL Framework

Herrera-Poyatos, David, Peláez-González, Carlos, Zuheros, Cristina, Tejedor, Virilo, Montes, Rosana, Herrera, Francisco

arXiv.org Artificial Intelligence

Large Language Models (LLMs) such as GPT -5, GEMINI, Claude, and LLaMA have become foundational tools in artificial intelligence (AI), achieving state-of-the-art performance in summarization, translation, reasoning, and dialogue. However, since LLMs are increasingly integrated in high-risk decision making in domains such as healthcare, law, and education, their lack of transparency raises urgent concerns for safety, accountability, and public trust [12]. The scale and complexity of these models, covering billions of parameters trained in opaque corpora, make their internal reasoning fundamentally inscrutable. This opacity creates barriers to responsible adoption, as users often lack meaningful ways to understand or challenge outputs. Without stakeholder-sensitive explanations, systems risk overtrust, misinterpretation, or outright rejection [11]. Explainable AI (XAI) for LLMs has therefore evolved beyond technical introspection [6]. The goal is not only to expose internal mechanisms but also to support human interaction, trust calibration, and decision assurance. As model behavior becomes more emergent and unpredictable [10], explanation systems must serve cognitive, functional, and ethical purposes simultaneously [7].


REFLEX Dataset: A Multimodal Dataset of Human Reactions to Robot Failures and Explanations

Khanna, Parag, Naoum, Andreas, Yadollahi, Elmira, Björkman, Mårten, Smith, Christian

arXiv.org Artificial Intelligence

--This work presents REFLEX: Robotic Explanations to FaiLures and Human EXpressions, a comprehensive mul-timodal dataset capturing human reactions to robot failures and subsequent explanations in collaborative settings. It aims to facilitate research into human-robot interaction dynamics, addressing the need to study reactions to both initial failures and explanations, as well as the evolution of these reactions in long-term interactions. By providing rich, annotated data on human responses to different types of failures, explanation levels, and explanation varying strategies, the dataset contributes to the development of more robust, adaptive, and satisfying robotic systems capable of maintaining positive relationships with human collaborators, even during challenges like repeated failures. I NTRODUCTION As robots become increasingly integrated into our everyday lives, from homes and workplaces to public spaces, the need to understand and improve human-robot interaction (HRI) has never been more critical. Despite significant advancements in robotics, they are still prone to failures, ranging from minor glitches to serious malfunctions.


Human-centered explanation does not fit all: The interplay of sociotechnical, cognitive, and individual factors in the effect AI explanations in algorithmic decision-making

Ahn, Yongsu, Lin, Yu-Run, Alikhani, Malihe, Cheon, Eunjeong

arXiv.org Artificial Intelligence

Recent XAI studies have investigated what constitutes a \textit{good} explanation in AI-assisted decision-making. Despite the widely accepted human-friendly properties of explanations, such as contrastive and selective, existing studies have yielded inconsistent findings. To address these gaps, our study focuses on the cognitive dimensions of explanation evaluation, by evaluating six explanations with different contrastive strategies and information selectivity and scrutinizing factors behind their valuation process. Our analysis results find that contrastive explanations are not the most preferable or understandable in general; Rather, different contrastive and selective explanations were appreciated to a different extent based on who they are, when, how, and what to explain -- with different level of cognitive load and engagement and sociotechnical contexts. Given these findings, we call for a nuanced view of explanation strategies, with implications for designing AI interfaces to accommodate individual and contextual differences in AI-assisted decision-making.


Let people fail! Exploring the influence of explainable virtual and robotic agents in learning-by-doing tasks

Matarese, Marco, Rea, Francesco, Rohlfing, Katharina J., Sciutti, Alessandra

arXiv.org Artificial Intelligence

Collaborative decision-making with artificial intelligence (AI) agents presents opportunities and challenges. While human-AI performance often surpasses that of individuals, the impact of such technology on human behavior remains insufficiently understood, primarily when AI agents can provide justifiable explanations for their suggestions. This study compares the effects of classic vs. partner-aware explanations on human behavior and performance during a learning-by-doing task. Three participant groups were involved: one interacting with a computer, another with a humanoid robot, and a third one without assistance. Results indicated that partner-aware explanations influenced participants differently based on the type of artificial agents involved. With the computer, participants enhanced their task completion times. At the same time, those interacting with the humanoid robot were more inclined to follow its suggestions, although they did not reduce their timing. Interestingly, participants autonomously performing the learning-by-doing task demonstrated superior knowledge acquisition than those assisted by explainable AI (XAI). These findings raise profound questions and have significant implications for automated tutoring and human-AI collaboration.


iSee: Advancing Multi-Shot Explainable AI Using Case-based Recommendations

Wijekoon, Anjana, Wiratunga, Nirmalie, Corsar, David, Martin, Kyle, Nkisi-Orji, Ikechukwu, Palihawadana, Chamath, Caro-Martínez, Marta, Díaz-Agudo, Belen, Bridge, Derek, Liret, Anne

arXiv.org Artificial Intelligence

Explainable AI (XAI) can greatly enhance user trust and satisfaction in AI-assisted decision-making processes. Recent findings suggest that a single explainer may not meet the diverse needs of multiple users in an AI system; indeed, even individual users may require multiple explanations. This highlights the necessity for a "multi-shot" approach, employing a combination of explainers to form what we introduce as an "explanation strategy". Tailored to a specific user or a user group, an "explanation experience" describes interactions with personalised strategies designed to enhance their AI decision-making processes. The iSee platform is designed for the intelligent sharing and reuse of explanation experiences, using Case-based Reasoning to advance best practices in XAI. The platform provides tools that enable AI system designers, i.e. design users, to design and iteratively revise the most suitable explanation strategy for their AI system to satisfy end-user needs. All knowledge generated within the iSee platform is formalised by the iSee ontology for interoperability. We use a summative mixed methods study protocol to evaluate the usability and utility of the iSee platform with six design users across varying levels of AI and XAI expertise. Our findings confirm that the iSee platform effectively generalises across applications and its potential to promote the adoption of XAI best practices.


Effects of Explanation Strategies to Resolve Failures in Human-Robot Collaboration

Khanna, Parag, Yadollahi, Elmira, Björkman, Mårten, Leite, Iolanda, Smith, Christian

arXiv.org Artificial Intelligence

Despite significant improvements in robot capabilities, they are likely to fail in human-robot collaborative tasks due to high unpredictability in human environments and varying human expectations. In this work, we explore the role of explanation of failures by a robot in a human-robot collaborative task. We present a user study incorporating common failures in collaborative tasks with human assistance to resolve the failure. In the study, a robot and a human work together to fill a shelf with objects. Upon encountering a failure, the robot explains the failure and the resolution to overcome the failure, either through handovers or humans completing the task. The study is conducted using different levels of robotic explanation based on the failure action, failure cause, and action history, and different strategies in providing the explanation over the course of repeated interaction. Our results show that the success in resolving the failures is not only a function of the level of explanation but also the type of failures. Furthermore, while novice users rate the robot higher overall in terms of their satisfaction with the explanation, their satisfaction is not only a function of the robot's explanation level at a certain round but also the prior information they received from the robot.


Behaviour Trees for Creating Conversational Explanation Experiences

Wijekoon, Anjana, Corsar, David, Wiratunga, Nirmalie

arXiv.org Artificial Intelligence

This paper presented an XAI system specification and an interactive dialogue model to facilitate the creation of Explanation Experiences (EE). Such specifications combine the knowledge of XAI, domain and system experts of a use case to formalise target user groups and their explanation needs and to implement explanation strategies to address those needs. Formalising the XAI system promotes the reuse of existing explainers and known explanation needs that can be refined and evolved over time using user evaluation feedback. The abstract EE dialogue model formalised the interactions between a user and an XAI system. The resulting EE conversational chatbot is personalised to an XAI system at run-time using the knowledge captured in its XAI system specification. This seamless integration is enabled by using Behaviour Trees (BT) to conceptualise both the EE dialogue model and the explanation strategies. In the evaluation, we discussed several desirable properties of using BTs over traditionally used STMs or FSMs. BTs promote the reusability of dialogue components through the hierarchical nature of the design. Sub-trees are modular, i.e. a sub-tree is responsible for a specific behaviour, which can be designed in different levels of granularity to improve human interpretability. The EE dialogue model consists of abstract behaviours needed to capture EE, accordingly, it can be implemented as a conversational, graphical or text-based interface which caters to different domains and users. There is a significant computational cost when using BTs for modelling dialogue, which we mitigate by using memory. Overall, we find that the ability to create robust conversational pathways dynamically makes BTs a good candidate for designing and implementing conversation for creating explanation experiences.


Explaining Classifications to Non Experts: An XAI User Study of Post Hoc Explanations for a Classifier When People Lack Expertise

Ford, Courtney, Keane, Mark T

arXiv.org Artificial Intelligence

Very few eXplainable AI (XAI) studies consider how users understanding of explanations might change depending on whether they know more or less about the to be explained domain (i.e., whether they differ in their expertise). Yet, expertise is a critical facet of most high stakes, human decision making (e.g., understanding how a trainee doctor differs from an experienced consultant). Accordingly, this paper reports a novel, user study (N=96) on how peoples expertise in a domain affects their understanding of post-hoc explanations by example for a deep-learning, black box classifier. The results show that peoples understanding of explanations for correct and incorrect classifications changes dramatically, on several dimensions (e.g., response times, perceptions of correctness and helpfulness), when the image-based domain considered is familiar (i.e., MNIST) as opposed to unfamiliar (i.e., Kannada MNIST). The wider implications of these new findings for XAI strategies are discussed.


Explanation Strategies as an Empirical-Analytical Lens for Socio-Technical Contextualization of Machine Learning Interpretability

Benjamin, Jesse Josua, Kinkeldey, Christoph, Müller-Birn, Claudia, Korjakow, Tim, Herbst, Eva-Maria

arXiv.org Artificial Intelligence

During a research project in which we developed a machine learning (ML) driven visualization system for non-ML experts, we reflected on interpretability research in ML, computer-supported collaborative work and human-computer interaction. We found that while there are manifold technical approaches, these often focus on ML experts and are evaluated in decontextualized empirical studies. We hypothesized that participatory design research may support the understanding of stakeholders' situated sense-making in our project, yet, found guidance regarding ML interpretability inexhaustive. Building on philosophy of technology, we formulated explanation strategies as an empirical-analytical lens explicating how technical explanations mediate the contextual preferences concerning people's interpretations. In this paper, we contribute a report of our proof-of-concept use of explanation strategies to analyze a co-design workshop with non-ML experts, methodological implications for participatory design research, design implications for explanations for non-ML experts and suggest further investigation of technological mediation theories in the ML interpretability space.